moelectric lines and under the silicon ni- 
tride infrared absorber. After the detec- 
tor structure is fabricated, the sacrificial 
layers are removed, typically by etching 
in an oxygen plasma. The removal of the 
sacrificial layers is what provides the ther- 
mal isolation mentioned above. 

The design facilitates maximization of 
the number of thermoelectric legs to in- 
crease the responsivity and the electrical 
impedance of the detector. Using 2-pm 
widths and 2-pm spacings of thermoelec- 
tric lines, it is possible to place about 11 
thermocouples under a 50-pm-wide pixel. 

Absorption of infrared radiation is en- 
hanced by use of a quarter-wave cavity. In 


each pixel, a thin layer of metal on the 
silicon nitride layer constitutes a front 
absorber, while the thermoelectric legs 
and interconnecting wires, together, 
constitute a back-side mirror. 

At the time of reporting the infor- 
mation for this article, partially com- 
pleted detectors (lacking the silicon 
nitride absorbers) of 100-pm pixel size 
had been built and tested. The results 
of the test indicate a pixel resistance 
of 250 kQ, responsivity of 1.5 kV/W, 
response time of 1.7 ms, and detectiv- 
ity (D*) of 2.4 x 10 8 cm-Hz 1/2 /W. Im- 
provements are ongoing. 
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Cascade Back-Propagation Learning in Neural Networks 

This method would be implemented in VLSI circuitry. 

NASA’s Jet Propulsion Laboratory, Pasadena, California 


The cascade back-propagation (CBP) 
algorithm is the basis of a conceptual de- 
sign for accelerating learning in artificial 
neural networks. The neural networks 
would be implemented as analog very- 
large-scale integrated (VLSI) circuits, 
and circuits to implement the CBP algo- 
rithm would be fabricated on the same 
VLSI circuit chips with the neural net- 
works. Heretofore, artificial neural net- 
works have learned slowly because it has 
been necessary to train them via soft- 
ware, for lack of a good on-chip learning 
technique. The CBP algorithm is an on- 
chip technique that provides for contin- 
uous learning in real time. 

Artificial neural networks are trained 
by example: A network is presented 
with training inputs for which the cor- 
rect outputs are known, and the algo- 
rithm strives to adjust the weights of 
synaptic connections in the network to 
make the actual outputs approach the 
correct outputs. The input data are 
generally divided into three parts. Two 
of the parts, called the “training” and 
“cross-validation” sets, respectively, 
must be such that the corresponding 
input/output pairs are known. During 
training, the cross-validation set en- 
ables verification of the status of the 
input-to-output transformation learned 
by the network to avoid overlearning. 
The third part of the data, termed the 
“test” set, consists of the inputs that are 
required to be transformed into out- 
puts; this set may or may not include 
the training set and/ or the cross-valida- 
tion set. 


Proposed neural-network circuitry for 
on-chip learning would be divided into 
two distinct networks; one for training 
and one for validation. Both networks 
would share the same synaptic weights. 
During training iterations, these weights 
would be continuously modulated ac- 
cording to the CBP algorithm, which is 


so named because it combines features of 
the back-propagation and cascade-corre- 
lation algorithms. Like other algorithms 
for learning in artificial neural networks, 
the CBP algorithm specifies an iterative 
process for adjusting the weights of 
synaptic connections by descent along 
the gradient of an error measure in the 



Validating Network Training Network 


Figure 1 . The Cascade Back-Propagation Algorithm provides the theoretical basis for design of an ana- 
log neural network that learns rapidly. 
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Figure 2. The Cascade Configuration of connections to added hidden neurons helps to accelerate con- 
vergence on the desired state of learning. 


vector space of synaptic-connection 
weights. The error measure is usually a 
quadratic function of the differences be- 
tween the actual and the correct outputs. 

The CBP algorithm (see Figure 1) be- 
gins with calculation of the weights be- 
tween the input and output layers of neu- 
rons by use of a pseudo-inverse technique. 
Then learning proceeds by gradient de- 
scent with the existing neurons as long as 
the rate of learning remains above a spec- 
ified threshold level. When the rate of 
learning falls below this level, a new hid- 
den neuron is added. When the quadratic 
error measure has descended to a value 
based on a predetermined criterion, the 
rate of learning is frozen. Thereafter, the 
network keeps learning endlessly with the 
existing neurons. 

Figure 2 illustrates the cascade aspect 
of the CBP algorithm. To each newly 
added hidden neuron there are not 
only weighted connections from all the 
inputs but also a new dimension of in- 
puts from the previous hidden neu- 
rons. The cascade aspect provides two 
important benefits: (1) it enables the 
network to get out of local minima of 
the quadratic error measure and (2) it 
accelerates convergence by eliminating 
the waste of time that would occur if 
gradient descent were allowed to occur 
in many equivalent subspaces of synap- 
tic-connection-weight space. The cas- 
cade scheme concentrates learning 


into one subspace that is a cone of a hy- 
percube. 

The gradient descent involves, among 
other things, computation of derivatives 
of neuron transfer curves. The proposed 
analog implementation would provide 
the effectively high resolution that is 
needed for such computations. Provi- 
sions for addition of neurons at learning- 
rate-threshold levels could be made eas- 
ily in hardware. 
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